Adaptive Sparseness Using Jeffreys Prior
نویسنده
چکیده
In this paper we introduce a new sparseness inducing prior which does not involve any (hyper)parameters that need to be adjusted or estimated. Although other applications are possible, we focus here on supervised learning problems: regression and classification. Experiments with several publicly available benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms support vector machines and performs competitively with the best alternative techniques, both in terms of error rates and sparseness, although it involves no tuning or adjusting of sparsenesscontrolling hyper-parameters.
منابع مشابه
Adaptive Sparseness for Supervised Learning
The goal of supervised learning is to infer a functional mapping based on a set of training examples. To achieve good generalization, it is necessary to control the “complexity” of the learned function. In Bayesian approaches, this is done by adopting a prior for the parameters of the function being learned. We propose a Bayesian approach to supervised learning, which leads to sparse solutions;...
متن کاملWavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior
The sparseness and decorrelation properties of the discrete wavelet transform have been exploited to develop powerful denoising methods. However, most of these methods have free parameters which have to be adjusted or estimated. In this paper, we propose a wavelet-based denoising technique without any free parameters; it is, in this sense, a "universal" method. Our approach uses empirical Bayes...
متن کاملBayesian Learning of Sparse Classifiers
Bayesian approaches to supervised learning use priors on the classifier parameters. However, few priors aim at achieving “sparse” classifiers, where irrelevant/redundant parameters are automatically set to zero. Two well-known ways of obtaining sparse classifiers are: use a zero-mean Laplacian prior on the parameters, and the “support vector machine” (SVM). Whether one uses a Laplacian prior or...
متن کاملMultichannel Adaptive Filtering with Sparseness Constraints
The performance of adaptive filtering can be enhanced by incorporating prior system knowledge. In this paper, we systematically consider regularization strategies exploiting sparseness for the identification of acoustic room impulse responses specifically for multichannel systems. Due to the additional dimensions in the multichannel case, a structured regularization appears to be a natural choi...
متن کاملReliable Dual Tensor Model Estimation in Single and Crossing Fibers Based on Jeffreys Prior
PURPOSE This paper presents and studies a framework for reliable modeling of diffusion MRI using a data-acquisition adaptive prior. METHODS Automated relevance determination estimates the mean of the posterior distribution of a rank-2 dual tensor model exploiting Jeffreys prior (JARD). This data-acquisition prior is based on the Fisher information matrix and enables the assessment whether two...
متن کامل